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Device with a human-machine interface. 
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The invention relates to a device with a human-machine interface. 
From the German Offenlegungsschrift DE 3700913 a user control 
interface for a computer device is known, in which the control organs are keyboard 
keys, which can be actuated by pressing. Images corresponding to the keys are dis- 
5 played together on a display screen, providing an identification of the function associ- 
ated with each key. 

In order to press a key, the user has to take the prior action of placing a 
finger on the key, and the device has means to detect this prior action. Upon placing the 
finger on the keyboard, the corresponding key on the display is highlighted, for exam- 
10 pie by changing its color. Upon pressing, a function of the key is effected. In this way, 
after touching the key, the user is given the option of giving a follow-up to the 
impending actuation by pressing the key, or, if an undesired key is touched, removing 
the finger without pressing the key. Thus, the physically impending actuation (touching) 
is a precondition for actuation (pressing), but the actuation need not inevitably follow 
15 the impending actuation. 

In this way, the user can operate the keyboard and check which key is 
about to be actuated without having to divert the eyes from the display screen. 

The known device has the problem that it occupies at least part of the 
^abriees^-output interface, even when no keys are to be actuated. In this way it reduces 
20 the amount of other information that can be passed by the output interface. It also 

diverts th^if^r^^tention without useful purpose. Apart from reducing the capacity for 
communication with the user, this may even be dangerous, for example in vehicles, like 
cars, (motor-)bicycle^eteeSera; which have many systems pertaining to them, that is, 
systems that are operable under normal driving circumstances. Some of ^ es ^^^^ 
25 like for cxample-tiie lighting system, are essential for the driving vehicl^o&ere, like 
-fef-f Ya mpl cheating systems, car radios, telephones, navigation systems or cassette 
player^pe supporting function. Such systems may be operated while the 
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vehicle is being driven by actuating selected control organs. For safety reasons, it is 
desirable that the driver should not need to avert the eyes from the road in order to find 
a control organ. At the same time, the risk of erroneous control actions due to selection 
I of a wrong control organ, which is significant if the driver has to grope for the control 

'if 

5 organs without looking, has to be minimized. 
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Amongst others, it is an object of the invention to provide a device with a 
human-interface which will not unduly burden the eyes of the user, and yet will also 
minimize the risk of activation of the wrong control functions. 

10 

To attain the object, there is provided a device with a human-machine 
(j interface, the device comprising an actuable control organ for selectably controlling a 

™! function of the system, the device comprising means for detecting a user action 

itJ 

C8 representative for physically impending actuation of the control organ, the device having 

15 means for, upon^aitf detecting, temporarily disclosing via output means, an indication 
of the selected function to the user. 

Here "disclosing via output means" is used in its meaning of "making 
perceptible what was not perceptible immediately before via the output means" to the 
user. 

20 The device^with a human-machine interface arranged according to the 

invention, is designed to ensure that actuation by the user is the result of a coherent 
movement, comprising reaching for the control organ and actuating it. Accordingly, "a 
user action representative of impending physical actuation" is meant to refer to any 
stage or result of such a coherent movement prior to actual actuation. This user action 
25 serves as a signal that further progression of the movement will lead to actual actuation. 
Upon detecting such impending actuation, the device with a human-machine interface 
according to the invention brings the function of the control organ temporarily to the 
attention of the user by disclosing an indication of that function. This indication may be 
the name of the function or simply a distinctive warning signal. If the indication corres- 
30 ponds to the one desired by the user, the user will proceed with the coherent movement 
and actuate the control organ, the actuation being effected for example by pressing, or 
turning, or holding the control organ for some time. If the indicated function is not 
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desired, the user will not proceed to the actual actuation stage, thus averting execution 
of an undesired function. 

Since the device with a human-machine interface is arranged such that the 
indication is temporary, some time after the actuation the indication will no longer be 
5 perceptible. Thus the users' attention will not be distracted for an indefinite amount of 
time. 

In principle, all control organs of a device with a human-machine 
interface may be provided according to the invention, but in practice it may suffice to 
arrange only potentially damaging or irreversible control functions, like switching off 

10 headlights of a car or detuning a car radio, in this way. 

An embodiment of the device according to the invention is characterized, 
in that the output means are general purpose output means. General purpose output 
means are usable for communicating more than one type of item of information, like 
indication of different functions, via the same communication channel to the user. For 

15 example, the same spot on a display screen may be used for disclosing indications of 
different functions, depending on the function for which impending actuation is 
detected. 

An embodiment of the user control device with a human-machine 
interface according to the invention, is characterized, in that the output means comprise 
20 means for disclosing the indication with an audible message. This has the additional 
advantage that the indication will be perceptible no matter where the user is looking: the 
user is not visually burdened by the device with a user control interface at all. An 
audible message moreover intrinsically only discloses information temporarily during 
the time it is uttered. 

25 An embodiment of the device according to the invention is characterized, 

in that it is a device pertaining to a vehicle. Especially in vehicles, like cars, 
motorcycles etc. it is advantageous that the users' eyes are not unduly drawn from the 
road. 

Another embodiment of the device with a human-machine interface accor- 
30 ding to the invention is characterized, in that the output means comprise a head up 

display, arranged for temporarily projecting the indication upon a windscreen. Thus, the 
user need not divert from the normal (drivers) viewing direction of the eyes to verify 



PHN 13.726 




27.03.1992 



the control function. 

In another embodiment of the device with a user control interface 
according to the invention, the control organ is provided outside the normal line of sight 
from a drivers position. The control organs may be provided for example with a cover, 
5 or placed adjacent to the drivers seat or underneath it, or be made undistinguishable 
from their surroundings. This has the advantage that the user will not be tempted to 
avert the eyes from the road. Additionally, it reduces the number of controls visible on 
the dashboard, which is advantageous for aesthetic and ergonomic reasons. Moreover, 
in automobiles, this makes it more difficult for would-be thieves to evaluate the features 

10 of a parked car. 

In a further embodiment of the device with a human-machine interface 
according to the invention, the control organ is part of a coherently placed group of 
control organs for activating respective functions from a function group, the indication 
being indicative of said function group as a whole. This is advantageous because it helps 

15 the user to get an overall indication of which group of control organs is being 
approached. Of course, in this case the means for detecting the user action may be 
provided detecting impending actuation of further control organs in the group as well 
and not just for detecting impending actuation of only one control organ. When there 
are several groups of control organs, indication of the group being approached will aid 

20 in finding the right group. When the function group is reassignable (the same group of 
control organs for example being assignable to control of a car radio or to control of a 
car telephone) this will moreover serve to inform the user whether the desired group of 
functions is assigned. 

Accordingly, in another embodiment of the device with a human-machine 

25 interface according to the invention, it comprises means for redefining the function 
group and the respective functions. 

In an embodiment of the device with a human-machine interface 
according to the invention, device with a human-machine interface comprises means for 
detecting a first and second stage of said user action, the device with a human-machine 

30 interface being arranged such that the first stage is representative of physically 
impending entry into the second stage, the second stage being representative of 
physically impending actuation of the control organ, the device with a human-machine 
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interface comprising means for, upon detecting the first stage, temporarily disclosing 
said indication, and upon detecting the second stage, disclosing, via the general purpose 
output means, a further indication which, within the group of control organs, is specific 
for the control organ. In this way the advantages of the group indication and the single 
5 control organ indication are combined. 

In a further embodiment of the device with a user control interface accor- 
ding to the invention, it is arranged for effecting said disclosing only when, after 
detecting said user action, actuation of the control organ is not detected within a 
predetermined period of time. In this way the user can avoid being burdened by the 

10 disclosing through acting quickly, for example within a predetermined time of a few 
tenths of a second. This is convenient for example for experienced user, or for a user 
which is already confident about the function of the control organ after trying a 
neighboring control organ. 

In another embodiment of the device with a user control interface 

15 according to the invention it is arranged for disabling the disclosing when actuation of 
the control organ is detected. In this way, once the decision to execute the control 
function is effected, the user is not burdened longer than necessary by the disclosing. 

In a further embodiment of the device with a user control interface accor- 
ding to the invention, it is arranged for disclosing a further indication of the function 

20 when actuation of the control organ is not detecte<L=tfee-within a further predetermined 
period of time after detecting said user action. Thus a hesitant user can be helped, for 
example by providing help information explaining the function of the control organ; a 
useful time interval before providing help information is in the order of several seconds. 

In an embodiment of the device with a user control interface according to 

25 the invention, the control organ is arranged for analog actuation of the function. Analog 
control settings tend to be hard to reproduce and should preferably not be altered 
inadvertently. Hence, for example a tuning dial, or a loudness control knob may be 
arranged for disclosing indication of their function in order to prevent inadvertent mani- 
pulation. 

30 In a further embodiment of the device with a user control interface accor- 

ding to the invention, it comprises proximity sensing means for generating a user action 
detection signal, the proximity sensing means being arranged to be responsive to appro- 
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ach of a human finger. This is advantageous in a vehicle because in such an environ- 
ment the user is usually reaching for a control organ, that is, unlike in using a computer 
keyboard, it is a problem to find the plurality of control organs itself. By providing for 
a response as soon as the user approaches the control organ it is made easier to find the 
control organs. 

The device with a user control interface according to the invention can 
advantageously be used in car radios and car telephones. By using their sound interface 
for disclosing the function of the control organs, an extra output interface need not be 
necessary. 

These and other advantageous aspects of the device with a human- 
machine interface according to the invention will be illustrated using the following 
Figures. 

15 Figures4a,/lb, Ij^liow an embodiment of the successive types of 

actuation of control organs ln^pr device with a human-machine interface^ 0 &ri&(_s 

Figure 2 sh6ws a schematic diagram of an implementation of the device 
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with a human-machine interface according to the invention 

si 



20 In Figures la, lb, lc«af^-example-o£^u^^ group 10 of control 

organs 11, 12, 13 for a device^with a user control interface is shown. By way of exam- 
ple, the control organs are shown as push buttons which can be pushed by a finger 14. 
Conventionally, to select one button 12 from the plurality 10, the user must look at the 
control organs to ensure that the finger is placed on the desired button 12 from the 

25 plurality 10. . 

In vehicles, like cars, (motor)bicycle^eetera in which the user at the 
same time is responsible for controlling for example the steering wheel and the brakes, 
it is undesirable that the user should dive eyes to look at the plurality of control 
organs 10. Preferably, this should even be made impossible, by placing the control 

30 organs outside the normal line of sight from the drivers seat, for example under a 
cover. Yet, for instance to control a car radio, the user must reach for the control 
organs; pushing one button reached blindly, that is, without looking at it first, is likely 
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to cause errors, which may be hard to correct -for example in the case of detuning the 
car radio- or costly -for example in/i^^y^ a wrong telephone number on a car phone. 
An error may even be damaging or dangerous in case the control button pressed 
erroneously concerns the driving of the car. 
5 To prevent such errors, the device with a user control interface according 

to the invention provides, upon detection of a user action representative of impending 
actuation of a control organ, -for an indication of the function of that control organ in 
such a way that the user can still prevent the function from being effected by not 
pursuing the action beyond the stage that lead the interface to conclude that actuation 

10 was physically impending. To prevent the user from being unnecessarily distracted by 
the mechanism for giving this indication, the indication is made available via a standard 
output interface used for different types of messages. It is made available temporarily, 
only for a limited time after detection of impending actuation of the control organ, that 
is, it is made perceptible to the user only at an attempt to actuate a function. This - 

15 minimizes distraction to the user, as is essential in a vehicle. 

Of course a device with a human machine interface without normally 
perceptible control organs will be useful in other environments than vehicles as well. 

To achieve such an interface, the idea is that an actuating action by the user will 
have successive stages, the interface effecting the function of the control organ only in 

20 the final stage, and in earlier stages providing indication of the function of the control 
organ selected. Thus, the user may avoid effecting the function by selectively breaking 
off the action before the final stage is reached. 

To illustrate the principle, in Figures la, lb, lc, the successive stages are 
triggered by successive distances of approach of the finger 14 to the control organ 12. 

25 The distance of the finger may for example be sensed by a proximity sensor operating 

by means of light, sound or heat. The stage of physically impending actuation occurs in 

Figure la when the finger 14 passes the border indicated by line 15; when the device 

K 

with a human-machine interface detects this, it provides an indication of the function 
associated with the control organ 1Z. Actual actuation is effected in Figure lb when the 
30 finger 14 passes the border 17; this causes the device with a user control interface to 
effect the function associated with the control organ 12^ The impending actuation is a 
physical precondition for the user to perform actuation. 
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It may be convenient to provide means for detecting two or more stages 
of actuation of all control organs. However, often it will be more attractive to do so for 
control organs corresponding to important functions only or for functions which are not 
easily reversible. This may be more convenient for economic reasons or for giving a 
5 higher alerting effect to the indication. 

The control organs are often coherently placed in groups of control 
organs which control related functions. For example in an automotive environment, the 
controls for heating are placed together, the controls for the car radio are placed 
together. It may be desirable to report first the function of the group and then the 

10 function of a specific control organ before effecting that function. For example to help 
the user to find a desired group more quickly by giving a coarse indication of the 
position of the finger. A way to achieve this is illustrated in Figure lc, where an 
intermediate actuation stage is detected when the finger 14 passes the border 16. In an 
device with a human-machine interface comprising three borders 15, 16, 17, the first 

15 border 15 A might trigger function group indication, the second border 15, sp&ific 
control organ information.and the third border 17 might cause the function to be 
effect ** *** 

Function group indication is also particularly useful when the function of a group 
of control organs may be reassigned as a group, for example when the same control 

20 organs selectably either control a car telephone or a car radio. In this case the function 
group indication serves to warn the user when the function group must be reassigned. 

Although the invention was discussed above with respect to pushbuttons, 
the invention is applicable to any kind of control organs, such as for example 
potentiometers, either of the rotating or of the sliding variety. 

25 It will be appreciated that there exist various possibilities for 

implementing detection mechanism for actuation and impending actuation of control 
organs. The only essential point is that the mechanisms are arranged for detecting more 
than one physical stage in an action that may lead to actuation, and that these stages are 
arranged in such a way that, when used by a human user, successive stages are physi- 

30 cally preconditions for later stages, but that the later stage may also be foregone if the 
user decides to. 

In its conceptually simplest form this is achieved by providing individual 
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control organs with at least two sensing mechanisms, one for detecting the physically 
impending actuation, another for detecting the actual actuation, the one being sensitive 
to an earlier stage of approach of a finger than the other. However, for the purpose of 
the invention, the* there need not be a physical relation between means for detecting 
5 actuation ^physically impending actuation^! a particular control organ; 

1) the detecting mechanisms for actuation and impending actuation may be 
separated physically, that is, be located at geometrically different locations, 

2) the detecting mechanisms may be shared among control organs, 

3) several control organs may be provided as a single physical device. 

10 To illustrate these possibilities: 1), for example if a push button is used for detecting the 
actuation, and an optical sensor is used for detecting the impending actuation, there is 
no need to place the pushbutton and the optical sensor together, as long as they sense 
the finger in the same region. To illustrate 2), using touch sensitive buttons for 
detecting the impending actuation (that is, for example capacitive sensors, which are 

15 actuated by touching rather than by exerting force) and a force sensor to detect the 
second actuation, the force sensor may be shared by several control organs since 
touching of a touch sensor is sufficient to determine which control organ is to receive 
the actual actuation. This has the advantage that it is guaranteed that both the impending 
and actual actuation will automatically correspond to the same control organ. To illu- 

20 strate 3), one may use a common touch sensitive surface provided with means to 

determine the coordinates of a point where it is touched; the coordinate values are use&ip 
select one virtual control organ anc^Setect the impending or actual actuation for this 
control organ; in this way one sensor serves for a plurality of control organs. Since 
according to the invention the functions of the control organs are identified with 

25 separate messages, such a touch tablet would require no further distinguishing marks to 
identify the control organs. Thus, an automobile, or indeed any control interface in any 
application outside a vehicle, could be provided without visible control organs. 

There is also an ample choice of sensors. One implementation uses a 
proximity sensor for impending actuation (the presence of a proximate object being 

30 detected using reflected or absorbed light, sound, electric or magnetic fields etc.) and a 
push button for detecting the actual actuation. Alternatively, the control organ may be 
sensitive independently to pushing and touching (without pushing) so as to detect the 



to 



PHN 13.726 27.03.1992 



impending and actual actuation as a consequence of pushing and touching respectively. 
In another embodiment, the control organ is a normal control organ, for example a 
pushbutton, which serves to detect the impending actuation; the actual actuation is 
detected if the button is kept pushed during a predetermined period of time (that is, long 
5 enough for the user to realize a mistake from the indication triggered by the first actua- 
tion). 

In any case, several actuation signals, corresponding to different stages of 
actuation will have to be derived for the control organ. 

Figure 2 shows an embodiment of an device with a human-machine 

10 interface according to the invention. For simplicity, only one control organ 12 is shown, 
which has outputs 3 1,^3^43- for three signals corresponding to detection of the first 
and second stages of impending actuation and the actual actuation respectively (the 
second stage being optional). In the embodiment, the first detection output is coupled to 
a sound signal generation circuit 22, which in turn is coupled to a loudspeaker 20. 

15 When the impending actuation is detected, for example after crossing the line 15 in 
Figure la, the function of the control organ can thus be identified via the loudspeaker. 
In a vehicle, the use of such an audible indication has the advantage that it does not dis- 
tract the eyes of the userjhe from the road, and that no distracting signals for the user 
are present until a control organ is almost actuated. Preferably, the signal generator 22 

20 produces a spoken message, for example by replaying a recorded message or using a 
speech synthesizer, but other signals, such as warning tones which identify functions as 
dangerous may also be useful. 

Alternatively, one may use a display driver 22 and a head up display 
screen for projecting the information on the windscreen upon detection of impending 

25 actuation. 

Another output 32 of the control organ 12 is for signalling that the actual 
actuation is detected, which will effect the function of the control organ. The effecting 
is symbolized in Figure 2 by connection of the output 32 to an activating input of a 
subsystem 21 for function execution. 
30 Several elaborations to the device with a human-machine interface, which 

are not essential to its functioning, but may be added to give substantial improvements 
to its effectiveness in minimizing the amount of unnecessary distraction of the users 
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attention are also shown in Figure 2 and will now be discussed. 

First of all, it may be convenient to interrupt the indication of a function 
once the function is activated by effecting the actual actuation. This is symbolized in 
Figure 2 by a switch 23 controlled by the output for the second mode 32 between the 
5 signal generator and the loudspeaker. In this way, unnecessary messages are cut short; 
it may even be convenient to give a slight delay (of the order of a few tenths of a 
second) to the signal generation 22 in order to enable the unhesitating user to avoid any 
signalling at all. 

Another addition is a second function report generator 26, triggered by 
10 detection of a second stage in the action leading up to actuation by the second stage 
detection output 33. Such stages may for example be detected by detecting a finger at 
different, decreasing distances to the control organ, but any combination of the detection 
mechanisms described above may be used, as long as one mechanism will trigger at an 
earlier stage of the user action than the other. In combination with the first function 
15 report generator 22 the first and second stage detection may be used to provide a first 
function group indication after detection of the first stage of the action and secondly a 
specific function indication after detecting the second stage of the action. It may be 
convenient to arrange the device with a user control interface to interrupt the group 
information if the user activates the specific information. 
20 Furthermore, it may be convenient to provide for additional help 

information for an extremely hesitating user, by allowing an impending actuation signal 
to pass through a time duration thresholding circuit 24 for triggering a help information 

a? 

generating circui^. In this way, when the state of impending actuation persists, but is not 
followed up by actuation within, for example, a few seconds, additional help informati- 
25 on can be reported via the loudspeaker^. 

Although Figure 2 has been drawn in the form of a schematic diagram, 
isolating several functions in separate boxes, it will be appreciated that the part of the 
interface providing the links between the control organ 12 and the output 20 can also 
be implemented as a program in a computer system. 
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